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[57] ABSTRACT 

A distributed automated testing system is provided which is 
capable of being distributed over a network, such as the 
Internet, for testing hardware and software. A plurality of 
users operating computers interface to the automated testing 
system via user interfaces, which preferably are graphical 
user interfaces. Each user interface displays test parameter 
choices to the user from which the user may select test 
parameters relating to a test to be performed. The user 
interfaces generate data packets in response to selections by 
the users and output the data packets onto the network. The 
data packets output from the user interfaces comprise infor- 
mation relating to test parameters selected by the user, 
commands indicating that performance of a test is being 
requested, and an address of the location to which the packet 
is being sent. The data packets are routed to one or more 
dispatcher machines located on the network which are 
designated by the addresses contained in the data packets. 
Each of the dispatcher machines maintains a list of tests to 
be performed. The dispatcher machine designated by the 
address in the data packet receives the data packet and 
updates the list of tests to be performed. A plurality of test 
machines are in communication with the dispatcher 
machines via the network. When a test machine is available, 
the available test machine generates availability data packets 
which indicate that the test machine is available to perform 
a test. Each of the availability data packets contains an 
address of a dispatcher machine. These availability data 
packets are sent over the network and routed to the dis- 
patcher machine designated by the address contained in the 
availability data packet. 

16 Claims, 6 Drawing Sheets 
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DISTRIBUTED AUTOMATED TESTING relating to the new test machine, such as its operating system 

SYSTEM and identity. Also, once a test machine is added, no provision 

is made for automatically rescheduling tasks taking the new 

TECHNICAL FIELD OF THE INVENTION test machine into consideration. This reduces the flexibility 

The present invention relates to a method and apparatus 5 a " d adapubilily of the system. Furthermore if a task fails 

for testing software and hardware and, more particularly, to af !f Pf f °TT ?* ^"as begun or if a test machine 

a distributed, expandable automated testing system which after tasks have been ^heduled on it, all of the affected 

allows users operating computers to test software and hard- Usks w11 need t0 * rescheduled - 

ware on test machines which are in communication with the Another disadvantage of the system proposed by Sun 

users' computers via a network. 10 Microsystems is that it is not a distributed system. The 

system proposed by Sun Microsystems is a local area 

BACKGROUND OF THE INVENTION network (LAN) having a client/server arrangement which 

Tne software testing phase is a critical phase in the scripts to communicate instructions between the users' 

software development process. During the software devel- 1S ^«t machines and the central server and between the 

£ . , /• - 1 central server and the target machines. No provision is made 

opment process, the software testing phase occurs after the - „ . , ° , t , * 4 4 . 

i_ u j j * i . j • • for allowing users located remotely with respect to the 
software has been designed, implemented in a programming 4 4 . ° . *u » u • 
language, and tested to a limited degree. frur^he-testinl s ^ m ° access the system because no commun.- 
^phase^software^esters ffi test,me 3 software>exrensivel.y c to^ ? lUo ™ P"* 0 ? 1 ^1 and transport- 
^eWthTTthT^ftware^meet^ 20 the f** the test . ^ P"* 0 ** * Sun 
-intended*to-ffiefi. In order to accommodate simultaneous 2 ° M.crosystems also has no proven for automatically con- 
testing of several different software packages by several ^""S a ts f machl ' le , and ^tomaUcaUy msulhng an 
tested, multiple test machines are often implemented. Dif- °P««Ung system on a test machine Rather, the tes system 
feren. types of software packages may need to be tested on b J Su ° M'« 0 sys'ems only allows a user to select 
different types of test machines, such as, for example, test 25 a test mac , kn6 > 1k ^ ^ving the operatmg system required 
machines with different hardware configurations and/or dif- * £° r ^T** ' P ?• ? r ' ' \ ?u f 
feren. operating systems. When a large number of software havin ? ^ ce .f?7 operatmg system for testing the soft- 
„ „ • j . u c & ware is not available, the test cannot be executed, 
testers are required to share common resources for software 

testing, provisions must be made for scheduling the tests in Accordingly, a need exists for an automated testing sys- 

order to efficiently manage these shared resources. The 30 tem which * automatically expandable to allow new test 

efficient management of these shared resources may also machines to be easily and automatically added to the system, 

require that tests and the results of the tests be recorded so and which fe distributed such that the users and components 

that the tests can be used repeatedly if needed and so that the of the test s y stem ma V be distributed over any network, such 

results of the tests can be analyzed and subsequently used for as the Internet - 

comparison with the results of tests performed a. a later 35 SUMMARY OF THE INVENTION 

In an effort to maximize efficiency in the handling of test In accordance with the present invention, an automated 

scheduling and test execution, attempts have been made to testing system is provided which is capable of being dis- 

automate software testing by using a server to manage test tributed over a network, such as the Internet, for testing 

machines and to allocate test packages among the test 40 hardware and software. A plurality of users operating com- 

machines in accordance with a schedule. Sun Microsystems, puters interface to the automated testing system via user 

Inc. has proposed an automated task-based scheduler for use interfaces, which preferably are graphical user interfaces, 

with UNIX platform systems which allows users operating Each user interface displays test parameter choices to the 

"client" machines to schedule tests to be executed on user from which the user may select test parameters relating 

"target" machines. A central server receives a request from 45 to a software or hardware test to be performed. Each user 

a client machine to perform a task. The server maintains interface generates data packets in response to selections 

information relating to all currently scheduled tasks on all from the user and outputs the data packets onto the network, 

target machines in a "status" database. The server maintains Generally, the data packets output from the user interface 

information relating to the expected duration of each test comprise information relating to test parameters selected by 

package and other test package attributes in a "packages" 50 the user, commands indicating that performance of a test is 

database. When the server receives a request to perform a being requested, and an address of the location to which the 

task from a client machine, the server determines the loads packet is being sent. A dispatcher machine located on the 

on each of the target machines based on the expected network and designated by the address contained in the data 

duration of each test package and then schedules the task on packet receives the data packets and updates a list of tests to 

the target machine with the least current toad. A task file 55 be performed. 

created at the client machine and copied to the server a plurality of test machines are in communication with 

includes priority information relating to the task requested the dispatcher machine via the network. When^a^test 

by the client machine. The target machine selects a task to macjrin^jsjav^^ 

be performed based on this priority information. Once a task f a]|iuj^ 

is completed, the results are copied back to the server which 6a||II^ availability data 

compares them to a set of "golden results" and creates a packets contSns^riTa^ressof a dispatcher machine. These 

comparison report which is mailed back to the user that availability data packets are sent over the network and 

requested the test. routed to the dispatcher machine designated by the address 

One disadvantage of the system proposed by Sun Micro- contained in the availability data packet. Upon receiving an 

systems is that the system cannot be easily expanded. If a 65 availability data packet, the dispatcher machine determines 

test machine is added to the system, the "machines" database whether one or more of the tests on the list of tests 

will need to be manually updated to include information maintained by the dispatcher machine are capable of being 
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performed by the test machine which generated the avail- 
ability data packet. If one or more of the tests listed are 
capable of being performed by the available test machine, 
the dispatcher machine instructs the test machine to perform 
one of the tests. This aspect of the present invention renders 
the automated testing system extremely adaptable in that it 
allows the system to be easily expanded or contracted 
because the system does not pre-allocate tests, but rather 
allocates tests in response to requests for work sent from the 
test machines to the dispatcher machines. 

In accordance with the preferred embodiment of the 
present invention, once the dispatcher machine receives an 
availability data packet from a test machine, the dispatcher 
machine determines whether any of the tests contained on 
the list of tests maintained by the dispatcher machine is 
capable of being performed on the available test machine 
and, if so, prioritizes these tests and instructs the available 
test machine to perform the test with the highest priority. The 
test machine then performs the test. Preferably, the test 
machine obtains the test and any archives associated with 
the test to be performed from a library which is in commu- 
nication with the user interfaces and with the test machines 
via the network. However, the test can be obtained from any 
location which is in communication with the network. 

When requesting performance of a test, a user can query 
the library via the user interface for information relating to 
tests and select test parameters relating to a test to be 
performed based on the information provided by the library 
to the user interface. Information relating to the user's 
selection will be contained in the data packets output onto 
the network to the dispatcher machine. 

Each test machine has a launcher program installed 
thereon which provides the application program interface 
between the test machines and the dispatcher machine and 
between the library and the test machines. The launcher 
program is responsible for preparing the test machine to 
execute a test, obtaining the test and any archives associated 
with the test from the library or from some other location on 
the network, installing the test and any associated archives 
on the test machine, preparing the test and archives to be 
executed, causing the test machine to execute the test, and 
outputting the results of the test to a predetermined location. 
In accordance with the preferred embodiment of the present 
invention, the results of the tests are output to the dispatcher 
machine. The users can query the dispatcher machines via 
the user interfaces to obtain status information about the 
tests. 

In accordance with the preferred embodiment of the 
present invention, the automated testing system of the 
present invention comprises an installer machine which is 
capable of automatically configuring the boot ROM settings 
on a test machine and/or installing operating systems on a 
test machine. To accomplish this, a multiplexer/ 
demultiplexer is connected to an RS232 port or ethernet 
local area network (LAN) port of each of the test machines 
and to the network. The installer machine is in communi- 
cation with the dispatcher machines via the network. The 
installer machine receives a data packet from a dispatcher 
machine indicating that a particular test machine is to be 
reconfigured and/or have an operating system installed on it. 
In response to receiving the data packet, the installer 
machine causes the test machine to be reconfigured and/or 
installed via either the RS232 or LAN port of the particular 
test machine. 

Once the test machine has been reconfigured and/or 
installed, the launcher program and any other necessary 
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software is installed on the test machine. When the launcher 
program is installed, the launcher program will notify all of 
the dispatcher machines with which it is allowed to com- 
municate that the test machine is on the system. The notified 
dispatcher machine will then update a machines file con- 
tained in the dispatcher machine with information relating to 
the test machine, such as the address of the test machine, its 
hardware configuration, which users are allowed to use it, 
and whether it can have its boot ROM reconfigured and/or 
have a new operating system installed on it. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram functionally illustrating the 
automated testing system of the present invention. 

FIG. 2 is a block diagram functionally illustrating the 
automated testing system of FIG. 1 further comprising an 
installer for installing operating systems on the test 
machines shown in FIG. X. 

FIG. 3 is a flow chart demonstrating the interactions 
between the launcher and the dispatcher of the automated 
testing system illustrated in FIG. 1. 

FIG. 4 is a flow chart demonstrating the interactions 
between the launcher and the librarian of the automated 
testing system illustrated in FIG. 1. 

FIG. 5 is a flow chart demonstrating the interactions 
between the user, the requester and the librarian of the 
automated testing system illustrated in FIG. 1. 

FIG. 6 is a flow chart demonstrating the interactions 
between the dispatcher and the launcher of the automated 
testing system illustrated in FIG. 1. 

FIG. 7 is a block diagram functionally illustrating the file 
locking process which takes place in the dispatcher of the 
automated testing system shown in FIG. 1. 

FIG. 8 is a block diagram functionally illustrating the 
connections between the installer and the test machines of 
the automated testing system shown in FIG. 1. 

DETAILED DESCRIPTION OF THE 
INVENTION 

I. Overall Discussion of the Automated Testing 
System 

FIG. 1 illustrates a functional block diagram of the 
automated testing system 10 of the present invention in 
accordance with a first embodiment. The lines interconnect- 
ing the components of the automated testing system 10 are 
intended to functionally illustrate communication between 
certain components of the system and between certain 
components and the users of the system and are not neces- 
sarily intended to indicate physical connections. In accor- 
dance with the preferred embodiment of the present 
invention, the system 10 is capable of being distributed over 
a network, such as the Internet. The lines shown in FIGS. 1, 
2 and 8 are merely intended to illustrate communication over 
the network. 

Generally, the automated testing system 10 allows users 
(not shown) operating computers 5 to test software and 
hardware on test machines 23. When hardware is being 
tested, the hardware will be comprised by the test machines 
23, whereas when software is being tested, the software will 
be installed on the test machines 23. Since the operations of 
the system 10 are essentially the same for testing both 
hardware and software, in the interest of brevity, the opera- 
tions of the system of the present invention will be discussed 
only with respect to software testing. It will be apparent to 
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those skilled in the art the manner in which the system of the uted so that centralization of the system components is not 

present invention can accomplish hardware testing in view required. As stated above, the components of the automated 

of the discussion provided herein of the operations of the testing system of the present invention do not have to be 

system for testing software. decentralized and distributed. However, the ability to dis- 

The users interface with the automated testing system 10 5 tribute and decentralize the components of the automated 

via requesters 13, which preferably are graphical user inter- testing system of the present invention is emphasized herein 

faces (GUIs). In accordance with the preferred embodiment because this feature of the present invention enhances the 

of the present invention, each user computer 5 will contain expandability of the automated testing system and allows 

a requester 13 which will provide the user with a graphical users and the components of the automated testing system to 

display format from which the user can select specifics 10 be located anywhere provided they are in communication 

regarding software tests to be performed from a list of menu over a network or over a group of networks, 

items displayed on a display screen. Preferably, the GUI As shown in FIG. 1, the users are in communication with 

comprised by the requester 13 is designed using Motif, dispatchers 17 via requesters 13. Generally, when a job is to 

which is a user interface tool box produced by The Open be executed, i.e., when a software package is to be tested, the 

Group. However, it will be apparent to those skilled in the 15 requesters 13 send requests to the dispatchers 17. The 

art that any user interface which provides the functionality dispatchers 17 then assign a priority to the job in view of 

needed by the requester 13 is suitable for use with the other jobs they have already received. The dispatchers 17 

present invention. It will also be apparent to those skilled in then determine whether there are test machines 23 capable 

the art that the requesters 13 do not have to be located on the 0 f performing the job that are not already executing other 

user computers 5, but may be separate components which 20 jobs and that are running the correct operating system for 

interface with the user computers 5. The operations and performing the job. If there are, the dispatcher 17 sends a 

functionality of the requester 13 are described in more detail message to the launchers 18 running on those test machines 

below. 23 which tells those test machines 23 to wake up and request 

Network 31 is illustrated merely for the purpose of work. The launcher 18 is a program installed on the test 

demonstrating that the automated testing system of the 2 5 machine 23 which constitutes the interface between the test 

present invention is distributed. Users located remotely with machine 23 and the other components of the automated 

respect to the automated testing system can have access to testing system. 

and communicate with the automated testing system of the Once these test machines 23 have been awakened, the 

present invention to perform testing via network 31 which launchers 18 of the test machines 23 notify the dispatchers 

may be, for example, the Internet. Also, the individual 30 17 whenever they are available for job execution by sending 

components of the automated testing system 10, such as the "available" packets to the dispatchers 17. Generally, these 

dispatchers 17, test machines 23 and libraries 21, may be available packets function as the clock of the automated 

separated and distributed over a network, such as the Inter- testing system 10 for triggering state changes within the 

net. However, it should be noted that the automated testing automated testing system. When a dispatcher 17 receives an 

system of the present invention and the users thereof do not 35 available packet from a launcher 18 of a test machine 23, it 

have to be separated over an extensive network but may be looks at a list of jobs waiting to be executed, determines 

located in proximity to one another, such as over a LAN or which of those jobs can be executed on the available test 

over an intranet. machine 23, prioritizes the jobs which can be executed on 

In accordance with the preferred embodiment of the the available test machine, and then sends the job with the 

present invention, an ASCII-based communications protocol 40 highest priority to the available test machines 23. Therefore, 

is used for communicating information between users and (/lla^jobas.o^ljr^blnitte^dlo a TesfnTachine 23 in~response-to 

the automated testing system as well as between the com- .receiptJ^the-dispatcher-17-of-a request.for_wprk rx6m~trTe 

ponents of the automated testing system. The ASCII-based ^test%iachmes-23,>e.,-in response to receiving an available 

communications protocol of the present invention, which is jpacket? This process minimizSTrafEc on the network^and 
discussed in greater detail below, generates packets whicE-45-mmimizes_operatmg.systeminstallati6h7~~~^ ~~~~ 

contain instructions and data. In accordance with the pre- If upon receiving a request to perform a job, the dis- 

ferred embodiment of the present invention, these packets patcher 17 determines that there are no test machines 23 

are framed using the well known Transmission Control capable of performing the job which are not already per- 

Protocol Over Internet Protocol (TCP/IP). These frames are forming jobs, or that the are no test machines 23 capable of 

then transmitted between the user computers 5 and the 50 performing the job which have the correct operating system 

automated testing system and between the components of already installed on them, the dispatcher 17 will determine 

the automated testing system over the Internet. As is well which of the test machines 23 capable of performing the job 

known to those skilled in the art, the TCP/IP protocol is executing the lowest priority job. Generally, the lowest 

comprises the network layer and transport layer of the well priority job is killed temporarily to allow the higher priority 

known Open Systems Interconnect (OSI) model. 55 job to be executed. If there is no lower priority job already 

It will be apparent to those skilled in the art that any being executed by a test machine 23 capable of performing 

network and transport layer protocols are suitable for use the job, the dispatcher 17 will determine which test 

with the present invention. However, TCP/IP is preferred machines 23 are allowed to have operating systems installed 

because it is a widely-used communications protocol and its on them and, of those, which are capable of performing the 

use with the present invention facilitates the distribution of 60 job. The dispatcher 17 then awakens those test machines, 

the automated testing system of the present invention over When those test machines 23 become available, the launch - 

the Internet. The user computers 5 and the components of the ers 18 of those test machines 23 send available packets to the 

automated testing system will be identified by Internet dispatcher 17. Upon receipt of an available packet, the 

Protocol (IP) addresses. By using IP addresses for the system dispatcher 17 prioritizes the jobs which can be executed on 

components and the users of the system, in conjunction with 65 the test machine which sent the packet, causes an operating 

TCP/IP and with the ASCII-based communications protocol system to be installed on the test machine, and then sends the 

of the present invention, the system can be entirely distrib- highest priority job to the test machine. The interaction 
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between the launchers 18 of the test machines 23 and the 
dispatchers 17 is discussed in detail below with respect to 
FIG. 3. 

When a requester 13 receives a request from a user 
computer 5, the requester 13 determines whether the user 
has permission to obtain the test requested and, if so, 
provides a list of tests and associated archives to the user 
computer 5. The user then selects one or more of the tests 
and archives. The requester 13 then sends the corresponding 
test names, along with the user's ID, to the dispatcher 17. 
The dispatcher 17 goes through the process discussed above 
of awakening test machines capable of performing the job 
and, upon receiving an available packet from the launcher 18 
of a test machine, prioritizes the jobs which can be executed 
on the test machine and then sends an "execute-job" com- 
mand to the test machine 23. The launcher 18 running on the 
test machine 23 receives the "execute -job" command and 
then obtains the test and any associated archives from the 
library 21, prepares the test machine 23 to execute the test, 
and causes the test machine 23 to execute the test. Once the 
test machine 23 has completed the test, the launcher 18 
forwards the results of the test to the dispatcher 17 which 
then notifies the user via the requester 13 that the test results 
are available. The user then obtains the test results from the 
dispatcher 17 via the requester 13. Alternatively, the results 
of the test can be saved in a database (not shown) outside of 
the dispatcher 17. In this case, the user will obtain the test 
results from that database via the requester 13. 

It should be noted that the tests do not have to be obtained 
from the library 21. A test may be obtained from virtually 
any location provided the launcher 18 is provided with the 
path to use to obtain the test. For example, if the launcher 18 
is provided with the path on the Internet for obtaining the 
test to be executed, the launcher 18 can simply obtain the test 
from its location on the Internet. Also, the test may be sent 
from the requester 13 through the dispatcher 17 to the 
launcher 18, if so desired. However, in order to minimize 
traffic on the network, preferably the tests are not sent from 
the requesters 13 through the dispatchers 17 to the launchers 
18, but instead are obtained by the launcher 18 from the 
library 21 or some other location on the network. 

It should be noted that the flow of control of the auto- 
mated testing system of the present invention is from the test 
machines 23 to the dispatchers 17, and not vice versa. 
Therefore, rather than the dispatchers 17 querying the test 
machines 23 to determine whether an appropriate test 
machine 23 is available, the test machines 23 notify the 
dispatchers 17 when they are available for job execution. In 
this respect, the present invention is diametrically different 
from typical task managers, which generally pre-allocate 
jobs to test machines based on a schedule to maximize 
parallel processing of jobs. Contrary to typical task 
managers, rather than pre- allocating jobs, the dispatcher 17 
of the present invention allocates jobs on test machines 23 
in response to notifications from the test machines 23 that 
they are available. In essence, the test machines 23 ask the 
dispatchers 17 to send them work. This aspect of the present 
invention maximizes the expandability and adaptability of 
the automated testing system of the present invention in that 
test machines 23 can be added to and removed from the 
automated testing system at any time without complicating 
the distribution of jobs. 

Furthermore, test machines 23 can be added to and 
removed from the automated testing system 10 automati- 
cally without having to manually update a file in the dis- 
patcher 17 to inform the dispatcher 17 that a test machine 23 
has been added to or removed from the system. When a test 
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machine 23 is added to the automated testing system, the 
launcher 18 is installed on the test machine 23. The launcher 
18 notifies the dispatchers 17 with which it can communi- 
cate that the test machine 23 is on the system. The dispatch- 
ers 17 maintain a file of the test machines 23 which are on 
the system and attributes of each test machine 23, such as the 
type of hardware and operating system on the test machine 
23. If a test machine 23 is removed from the automated 
testing system 10, either intentionally or because of a failure 
of the test machine 23, no indication that the test machine 23 
is available will ever be received by the dispatcher 17. 
Therefore, the dispatcher 17 will never send a job to the 
removed test machine 23. On the other hand, if a test 
machine 23 is added to the automated testing system, the 
launcher 18 informs the dispatcher 17 that the test machine 
23 is on the system 10. When the test machine 23 becomes 
available it will notify the dispatcher 17 that it is available. 
Therefore, the automated testing system of the present 
invention automatically adapts as test machines 23 are added 
to or removed from the system 10. 

FIG. 2 illustrates the preferred embodiment of the auto- 
mated testing system 10 of the present invention wherein an 
installer 35 has been added to the system. For ease of 
illustration, the user computers 5, the network 31 and the 
launchers 18 are not shown and only one requester 13 is 
shown. In accordance with this embodiment of the present 
invention, the test machines 23 can be reconfigured and/or 
have operating systems (OSs) installed on them. A user can 
request execution of a test which requires that a new OS be 
installed on a test machine 23. Upon receiving the job 
request, the dispatcher 17 will notify the installer 35 that a 
particular test machine 23 is to have a particular OS installed 
on it. The installer 35 can also install operating systems on 
multiple test machines 23 simultaneously. In accordance 
with the present invention, an installer 35 is provided which 
interfaces with the test machines 23 via an RS232 connector 
or via a multiplexed array of RS232 connectors to allow the 
system to automatically install operating systems on one or 
more test machines. Alternatively, the installer 35 may 
interface with the test machines 23 via the LAN ports of the 
test machines 23 using, for example, an installer produced 
by Hewlett Packard called Ignite UX, as is known to those 
skilled in the art. In addition to installing an OS on a test 
machine 23, the installer 35 can also modify boot-ROM 
settings or re-boot the test machine 23. The installer 35 is 
discussed in more detail below. 

It should be noted that the automated testing system of the 
present invention is not limited with respect to the number 
of requesters 13, dispatchers 17, test machines 23, libraries 
21 or installers 35 which can be implemented with the 
system or with respect to the number of users who can use 
the system. It should also be noted that the automated testing 
system of the present invention is not limited with respect to 
the number of user computers 5 and requesters 13 which can 
interface with a single dispatcher 17 or with respect to the 
number of test machines 23 which can interface with a single 
dispatcher 17. In order to maximize efficiency in the sharing 
of resources, more than one test machine 23 can communi- 
cate with a single dispatcher 17 and more than one dis- 
patcher 17 can communicate with a single test machine 23. 
Multiple requesters 13 and multiple test machines 23 pref- 
erably will be in communication with library 21 in order to 
maximize efficiency in resource sharing. Also, the locations 
of the user computers 5 with respect to the automated testing 
system and the locations of components of the automated 
testing system with respect to each other are not geographi- 
cally limited. 
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It will be apparent to those skilled in the art that any 23. When a launcher is installed on a test machine, the 

network and any communications protocol for communicat- launcher will contain a list of the IP addresses of the 

ing information over the network is suitable for use with the dispatchers with which the launcher may communicate, 

automated testing system of the present invention. It will When a test machine is available, the launcher of the test 

also be apparent to those skilled in the art that the automated 5 machine will advertise the test machine as available by 

testing system of the present invention and the user com- sending "available" packets to the dispatchers on the list in 

puters 5 can be distributed over any network and even over a round robin fashion. In order to allow test machines to be 

a plurality of different networks. For example, the compo- reserved, or checked out, a dispatcher can command a test 

nents of the system of the present invention and the users machine to stop advertising itself as being available, 

thereof can be distributed over the Public Switched Tele- to The launcher 18 prepares the test machine 23 to perform 

phone Network (PSTN), over an Integrated Services Digital a test » causes the test machine 23 to execute the test, and then 

Network (ISDN), or over any combination of these net- leaves the test machine 23 in a predetermined state for 

works. receiving the next job. The launcher 18 running on a test 

As staled above, preferably the components of the auto- macbine 23 determines when the test machine 23 is avail- 
mated testing system of the present invention and the users 15 able to execute a job. Once the test machine 23 becomes 
thereof communicate with each other via the Internet. In ««l«ble. the launcher 18 sends an avadabk packet . tc .the 
order to accomplish this, instructions and data are encapsu- dispatcher 17, as indicated at block 38. The available 
lated in packets in accordance with the ASCII-based com- P acket Preferably comprises the IP address of the user s 
munitions protocol of the present invention and then these computer 5 the current time and date, information regarding 
packets are framed in accordance with the TCP/IP protocol 20 the test machine type, and informahon regarding the type of 
and sent over the Internet. Therefore, in accordance with the OS running on the test machine 23. Toe "available packet 
preferred embodiment of the present invention, each user of Preferably also contains information relating to whether the 
the automated testing system will have the ASCII-based test maclune 23 can be c ? ld installed, i.e.. whether a new 
communications protocol of the present invention as well as ^^J**? T be ,nstaUed on the f* m ? chlM 2i ' 
a TCP/IP utility installed on his or her computer 5 to allow 25 V* "available packet may also contain a list of users or a 

users to receive and transmit the automated testing system hst of Z™? 5 °[ users who are allowed t0 usc that P»^» 

packets. Similarly, each component of the automated testing test macnme 23* ^ ^ 

system 10, such as the dispatchers 17 and test machines 23, U P on receiving the '/available" packetrthe dispatcherl7 ^ 

will have the ASCII-based communications protocol of the determines whether it has r^iy^^rguest to execute a/ 

present invention as well as a TCP/IP utility installed 3 ° j^astodic^^^blo 

thereon to allow each component to receive and transmit the ; received a request to execute a job frpnUhe requester 1^ the] 

automated testing system packets. L dispatcher 17 sends an-"execute-nothing"-packet_to the^] 

r , -*u 4U e j u j- * c *u launcher 18, as indicated at block 40. The "execute-nothing" 

In accordance with the preferred embodiment of the r~T * ~ , j -m -TSj— * * -—^^-^ *n 

„ . nn ^ rat nanorn , a A f u Q AcriT packet includes the IP address of the test machine 23 as well 

present invention, each packet generated using the ASCII- ,/ t , t «-£■/"'.. « -i .i „ 

f j • ♦- « i c ,l . • 4- 35 as the current date and tune. If after receiving the "available 

based communications protocol of the present invention - rr^r — j.— ~rxr~ ■ tL . f »_ ■ j 

. . „ . .„ , r A . f n f, „ , . „ packet the dispatcher 17 determines that it has received a 

begins with a start keyword, is followed by a machines r w r , • l *u j- * u n i i *i-4f 

keyword, and ends with an "end" keyword. Each packet is r ^ ue f l to ? xe ° ute a J ob ; th f d ^ atchcr }\ lo °^ ?' » hsl of 

• \ c i- - t u u I- u • all jobs which are ready to be executed and determines 

comprised of one or more lines with each line beginning u- u e *u • u * ui c a *u 

• t . r , j j j- - tL 4 . mi * i4i- which of those jobs is capable of being performed on the 

with a keyword and ending with the following packet line. U1 t t J . 

Each packet line contains no more than 1024 characters. The 40 avallab J e f ' mach ' ne - ™? dls P atche 5 " then ?™ nX ™ 5 the 

first line of every packet contains the "start" keyword. The J ob * whicb ™ £ T ■ .l ^ 

A r t * A . « u „ t performed on the available test machine, and selects the job 

second line of every packet contains the machines key- r . u , , . . t , ' . „ * 

word The last line of every packet contains the "end" Wlth lhe hlgheSt pn0rity ' * mdlcated at block 41 ^ 

keyword. The "machines" keyword identifies the machine dis P atcher t^n determines whether the operating system 

■t i t . m j j r™. „ . t „ j (t id 45 running on the test machine 23 will need to be changed in 

sending the packet by its IP address. The "start and "end . * , 4 , . . . t , t U1 , A - Tr & , 

. , mm v * ■ fl . e • tUa ^ tam no tUa order to execute the job, as indicated at block 42. If not, the 

keywords are not saved in any files in the system as the , . t . . _ . J ' * • i . . i_ . * 

, t 1 1 , , tU 4 u 4i_ i _i dispatcher 17 sends an execute job packet to the launcher 

packets are sent through the system because these keywords * indicated at block 45 The "execute-iob" nacket 

are merely used for packet framing and for communicating "r 1 Q ™ f* 1 bioc f ltie execu ^*JOO packet 

. . J , . , j- * l .t i 4 u • 5 includes the IP address of the test machine 23, the current 

between machines to indicate where the packet begins and , , , , . , . , .„ . . ^ 

j . , , . * i . ,t i * -11 50 date and time, and a job identification indication. The 

ends. Any other keywords contained m the packet will „ * ■ u» 1 * i • . j * • • . « • ; j 

, J f*u execute -iob packet also includes test installation/setup and 

depend on the purpose of the packet. j r« , ^ U / 

r execution commands. These are commands which tell the 

Since the design of communications protocols is well launcher 18 how t0 mstall the test soflware 0Q the tes( 

known in the art, a detailed discussion of the ASCII-based mac hi ne 23, how to configure the environment variables, 

communications protocol of the present invention will not 55 and how t0 start the test software on lhe test machine 23 . 

be provided herein. It will be apparent to those skilled in the If at block 42 lfae ^ atcher 17 delermines that the 

art,inviewofthedisctissionoftheautomatedtestin^ u needs {Q be {h& 6]s hcj 17 

of the present invention provided herein, the manner in sends a ^ s „ {Q ^ Me / 3S M 

which a suitable commumcations protocol can be designed CQQ{a[ns ^ ip addregs of ^ ^ macMne 23 Qn which 

which meets the objectives of the present invention. 6Q flew os win be &$ ^ ag the QS version and ^ 

II. Launcher/Dispatcher Interactions as mdicated at block 43 " ^ e ^^taUer 35 will then instaU the 

new operating system on the test machine 23, as indicated at 

FIG. 3 is a flow chart depicting the interactions between block 44. Once the new operating system has been installed, 

the launcher 18 of a test machine 23 and one of the the dispatcher 17 sends the "execute-job" packet to the 

dispatchers 17. The launcher 18 is a daemon which consti- 65 launcher 18, as indicated by block 45. The launcher 18 then 

tutes the interface between the dispatcher 17 and the test causes the test machine 23 to execute the job, as indicated 

machine 23 and between the library 21 and the test machine at block 46. When the launcher 46 determines that the test 
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has been completed, as indicated at block 47, the launcher 18 
will forward the results to the dispatcher 17, as indicated at 
block 48. The results may then be saved in the dispatcher 17 
or in any other desired location. The results may also be 
reported by the dispatcher 17 to the user via the requester 13 
or the dispatcher 17 may inform the requester that the results 
are available and provide the requester 13 with the path for 
obtaining the results. 

Preferably, the launcher 18 of the test machine 23 sends 
status messages to the dispatcher 17 when (a) the test 
machine 23 begins executing the test, (b) once the test is 
complete and (c) after the job has been fully processed. The 
job is not fully processed until the test is complete and the 
test machine 23 has been left in a predetermined state (e.g., 
all necessary software has been removed from the test 
machine 23). 

III. Dispatcher/Launcher/Librarian Interactions 

FIG. 4 illustrates a flow chart which generally describes 
the functions of the launcher 18 upon receiving an "execute- 
job" packet from the dispatcher 17. Upon receiving an 
"execute -job" packet from the dispatcher 17, as indicated by 
block 52, the launcher 18 sends a "get-test" packet to the 
librarian, as indicated at block 53. The librarian is a daemon 
that controls the operations of library 21. The "get-test" 
packet comprises the test name and version, the archive 
version, and the IP address of the library 21. In reply to the 
librarian receiving the "get-test" packet, the library 21 sends 
a "test -install" packet to the launcher 18, as indicated at 
block 55. Alternatively, instead of sending the "test-install" 
packet to the launcher 18, the librarian may send a "status" 
packet to the launcher 18 indicating that the test requested is 
unavailable or cannot be sent. This "status" packet will 
comprise the IP address of the test machine 23, the current 
date and time, and a "bad" status indication. 

As stated above, the launcher 18 installs the test software, 
configures the environment variables that affect the test, and 
configures, initializes, and starts the test software. The 
"test-install" packet sent by the librarian to the launcher 18 
includes instructions that command the launcher 18 to 
perform all of these tasks, which are illustrated by blocks 
57-63. Also, if the library 21 contains an archive which is 
associated with the requested test, the launcher 18 will send 
a "get-archive" packet (step not shown) to the library 21 
specifying the archive requested, the IP address of the test 
machine 23 to which the archive is to be sent, and the test 
name and test version associated with the archive. In 
response to the "get- archive" command, the librarian will 
send an "archive -ins tall" packet (step not shown) to the 
launcher 18 comprising instructions which the launcher 18 
will use to install, configure and start the archive. If the 
archive requested cannot be sent, the librarian will reply 
with the "status" packet (step not shown) indicating that the 
status is "bad". 

Once the test and any associated archives have been 
installed and configured, the launcher 18 will command the 
test machine 23 to execute the test, as indicated at block 65. 
Once the test is complete, the launcher sends the results of 
the test to the dispatcher 17, as indicated at block 66, and 
removes software or performs any other actions necessary to 
leave the test machine in a predetermined state for receiving 
the next job, as indicated at block 68. Additionally, the test 
machine 23 can be commanded to send the test results to any 
machine by sending a "results-to" command to the test 
machine 23 which comprises the IP address of the machine 
to which the results are to be sent. Once the test has been 
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completed, the launcher 18 can remove or load software in 
order to leave the test machine 23 in a predetermined state. 
The "test-install" packet discussed above also contains 
instructions which tell the launcher 18 how to load or 
5 remove the software. 

IV User/Reguester/Library Interactions 

The interactions between the user and the requester 13 
and between the requester 13 and the dispatcher 17 will be 

1 described with reference to FIGS. 5 and 6. As stated above, 
the requester 13 preferably is a graphical user interface 
(GUI) residing on the users' computers 5. The GUI displays 
menu items to the user on a display screen. For example, the 
display screen may display a list of different types of test 
machines 23 from which the user may select either a suitable 
type of test machine 23 for the job to be executed or a 
specific test machine 23 having a particular IP address. The 
display screen may also display a list of different types of 
operating systems from which the user may select an appro- 
priate operating system for the job to be executed. The user 
may select a suitable test machine 23 and a suitable oper- 
ating system via the GUI. The requester 13 queries the 
library 21, or several different libraries, to cause a list of tests 
to be displayed to the user on the display screen. The user 
may then select a test in the same manner in which the user 
selects a test machine and an operating system. It will be 
apparent to those skilled in the art that the user interface can 
be designed and implemented to display virtually any 
desired menu items to provide users with a vast number of 

30 test parameter choices. 

In accordance with the preferred embodiment of the 
present invention, it is the responsibility of the requester 13 
to verify that the user has the appropriate permission to 

35 perform library operations. The permissions verification 
process is illustrated in FIG. 5. When the requester 13 
receives a job request from the user 5, as indicated by block 
69, the requester 13 queries the librarian by sending a 
"query -library -permissions" packet to the library 21, which 

40 comprises an indication of the user's identity, as indicated at 
block 72. In reply, the librarian causes a "library- 
permissions" packet to be sent to the requester 13 which 
specifies users and/or groups of users that have permission 
to interact with the library 21, as well as the permitted level 

45 of interaction, as indicated at block 73. In accordance with 
the preferred embodiment of the present invention, three 
levels of interaction are provided, namely, read permission, 
add permission, and remove permission. Users having per- 
mission to remove a test also have permission to add a test 

50 and read a test. Users having only permission to add a test 
also have permission to read a test, but do not have permis- 
sion to remove a test. Users having only permission to read 
a test do not have permission to add or remove a test. 
Users with read permission may, via the requester 13, 

55 query the librarian to determine which tests and archives are 
available and the location of the tests and archives. Users 
with add permission can, via the requester 13, add new 
versions of tests and archives to the library 21 . The requester 
13 can also specify default test parameters for the tests. 

60 Alternatively, the user may, via requester 13, override the 
default parameters and specify his or her own test param- 
eters. Users with remove permission can remove specific 
versions of tests and archives from the library 21. 

It should be noted that the librarian is a daemon that can 

65 run on any machine 23. Libraries 21 can also be on any 
machine 23 and a library 21 does not have to be on the same 
machine 23 as the librarian. Also, more than one librarian 
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can be queried to permit querying of multiple libraries 21. provide this information to the dispatcher 17. However, this 

Therefore, although one library 21 is shown in FIGS. 1 and would require updating the dispatcher 17 when tests are 

2, this is merely for ease of illustration. It will be apparent changed, added or removed from the libraries 21, which is 

to those skilled in the art that, as is the case with all of the undesirable. By allowing the requester 13 to specify these 

components of the automated testing system of the present 5 features, the dispatcher 17 is relieved of these types of 

invention, libraries 21 can be distributed and located virtu- processing tasks so that it is able to focus on scheduling and 

ally anywhere provided they are accessible to the requesters dispatching jobs to appropriate test machines 23 and receiv- 

13 and test machines 23. Preferably, the librarian is hosted ing and organizing the results from the test machines 23. It 

on the machine which comprises library 21. will be apparent to those skilled in the art that these tasks 

Assuming that a test to be performed has already been 10 may be allocated among the components of the automated 

placed in the library 21, the requester 13 queries the librarian testing system and/or among machines outside the auto- 

for test information by sending a "query-test-information" mated testing system in any desired manner in order to 

packet to the library 21, as indicated at block 76. The maximize performance and/or efficiency, 

"query-test-information" packet may contain information When the dispatcher~17 receives the^'submit-job'-packet 

requesting either a list of categories, a list of tests under a 15 fronrthe requester 13, the dispatcher 17 decodes:the:request 

specific category, a list of all tests, or information about a Var^serids an "acknowledge!^ ack^ 

specific test. The librarian replies to the request by sending ^which f contains a job-identification indication,^ indicated'at 

a "test-information" packet to the requester 13 which con- ^block 82rThis identification may be used by the" requester-13 

tains the information requested. For example, if the ^for-any query requests for information-relating toTthe job, 

requester 13 requested a list of test categories, the "test- 20_ S uch^as^he-stams"of-the-job. -The- dispatcher 17 then 

information" packet sent to the requester 13 will contain a determines„whether_thejob needs~tb~be~executed iEnmedi- 

list of categories from which the user may select the appro- atelyor at a later time i based'b^infoWatio"n"c^o^taihed : m theo 

priate category. If the user selects a particular category from ^submit-job" packets as -indicatedjit block 83 r-When'iCis 

the category list by, for example, placing the cursor over the ^time for Ihelfjbb to be executed, thedispatcher 17 wakes^up 

desired category by moving a mouse and by clicking the 25 the test machines^lnch areicapable.ot.executing.the job, as 

mouse button, the requester 13 will send another "query- r indicated at block 84. The interactionsjj etween th e Jaunch- 

test-information" packet to the librarian which contains an ers 18 and the dispatchers~17 then" take pla^e"inthe~manner 
instruction to the librarian to send a list of a tests under thaf~~~ discussed above with reference to FIGSr3"and"4"t 

specific category. The librarian will reply with a "test-*" ^Dis^h^ntemTorTerations 

information" packet which contains a list of the tests by test 30 v f 

name and which may also contain a description of the test, Tne operations of the dispatchers 17 will now be 

the category to which the test belongs and related categories. described in detail with respect to FIG. 7. The dispatchers 17 

The user may then, via the requester 13, select one or more are responsible for several tasks, such as job scheduling, job 

tests from the list and submit the job to the dispatcher 18, as management, communicating with requesters 13, installers 

indicated at block 77. 35 ^ an d launchers 18, keeping track of job text, and moni- 

In order for the requester 13 to submit the job to the torin 8 permissions. In order to accomplish these tasks, each 
dispatcher 17, the user's selection is embedded in a "submit- dispatcher is comprised of several daemons. Each daemon is 
job" packet and sent to the dispatcher 17. All packets responsible for handling all of these tasks. Each of the 
formatted in accordance with the communications protocol dispatchers 17 maintains several files which the dispatcher 
of the present invention contain the name of the machine 40 daemons read and write. The dispatchers 17 do not corn- 
sending the packet as well as the IP address of the machine. municate with each other. The files which are written and 
Preferably, the receiving machine always sends an acknowl- read b y the dispatcher daemons include a machines file 
edge packet to the sending machine upon receipt of the whlch maintains information about test machines, such as 
packet from the sending machine to inform the sending me operating system on the machine and whether the 
machine that the packet has been received and to provide the 45 machine can be cold installed, a permissions file which 
sending machine with other useful information, such as maintains information about user permissions, and a priori- 
whether the receiving machine can perform the received des file which maintains information about user priorities, 
command. anc * a j ob n * e wmcn maintains a list of jobs and the priority 

ranking for each job. 

V. Dispatcher/Requester Interactions so Each of lhe flles ma j at ained within a dispatcher may be 

FIG. 6 is a flow chart illustrating the tasks performed by accessed by all of the dispatcher daemons. Since each of the 

the dispatcher 17 after it has received a request from a dispatcher daemons is capable of writing these files, a 

requester 13 to run a test. Once the user has selected the test locking functionality is provided which allows a dispatcher 

or tests to be performed, the requester 13 sends a "submit daemon to open and lock a file to give that particular daemon 

job" packet to the dispatcher 17, as indicated by block 81. 55 exclusive access to the file to perform an operation. This 

The "submit job" packet contains the name and IP address locking functionality is exemplified by the block diagram 

of the requester 13, the location of the test and any associ- illustrated in FIG. 7. As shown in FIG. 7, several dispatcher 

ated archive in the library 21, the operating system needed daemons 101, 102 and 103 compete to open a file 105. If 

for the test machine 23 and the type of test machine 23 daemon 101 attempts to open file 105 before daemons 102 

needed to perform the test. It should be noted that although 60 and 103 attempt to open file 105, daemon 101 will be 

it is preferred that the requester 13 inform the dispatcher 17 granted exclusive access to file 105 until it is finished 

as to the type of test machine 23 needed for the test and the reading or writing the file 105. When daemon 101 is finished 

type of operating system needed on the test machine 23, it reading or writing file 105, file 105 is unlocked and which- 

is possible for the dispatcher 17 to maintain a file of the ever of daemons 102 and 103 attempted to access file 105 

names of the tests contained in the libraries, along with the 65 first is then granted exclusive access to file 105 until it has 

types of test machines and operating systems needed for finished reading or writing file 105 at which time file 105 is 

particular tests so that the requester 13 does not need to unlocked. In the example shown in FIG. 7, daemon 102 
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attempted to unlock file 105 before daemon 103 attempted 
to unlock the file. Therefore, daemon 102 will be granted 
exclusive access to file 105 before daemon 103, 

In order to prevent file 105 from being permanently 
locked in the event that a daemon dies after being granted 
access to the file 105, the daemons are designed to unlock 
the files in the event that they die after a file has been opened 
and locked. Since file locking is well known in the art, in the 
interest of brevity, a detailed discussion of the file locking 



installer 35 and the dispatcher 17 are each on the Internet 
and each have IP addresses. As is the case with the dis- 
patchers 17, the present invention is not limited with respect 
to the number of installers 35 comprised by the automated 
testing system. Therefore, although only one installer 35 is 
shown in FIG. 8, it will be apparent to those skilled in the 
art that the automated testing system of the present invention 
may contain many installers 35. 
It should be noted that the installer 35 of the present 



mechanism of the present invention will not be provided 10 invention is capable of simultaneously configuring several 



herein. It will be apparent to those skilled in the art the 
manner in which such a file locking and unlocking function 
can be implemented to achieve the goals of the present 
invention. 

The dispatcher of the present invention can be comprised 
of any type of computer. Most of the processing of the 
dispatchers is accomplished by software, i.e., by the dae- 
mons running on the dispatchers. Therefore, the computer 
used for the dispatcher should be capable of simultaneously 
running several daemon applications. Since the system of 
the present invention preferably is intended to communicate 
over the Internet, the computer used as the dispatcher should 
also be equipped to receive and send messages over the 
Internet. 

VII. Installer 

FIG. 8 is a block diagram which will be used to demon- 
strate the cold installing process of the present invention 
which is used to configure a test machine and/or install an 
operating system on the test machine. Normally, a computer, 
such as an IBM-compatible personal computer (PC), is 
configured manually by someone communicating with the 
computer via the computer console which is connected to 
the RS232 port of the computer. Under these conditions, the 
boot ROM, which may be, for example, BIOS, causes 
prompts to be output over the RS232 port to the display. The 
user sitting at the console types in appropriate replies on a 
keyboard to configure the hardware of the computer. Once 



test machines and simultaneously installing operating sys- 
tems on several test machines. The installer 35, the test 
machines 23 and the dispatcher 17 are all in communication 
with one another over network 109, which is the Internet in 
is accordance with the preferred embodiment. The test 
machines 23 interface with network 109 via network 
multiplexer/demultiplexer 108. When dispatcher 17 receives 
a command to install an operating system on one of the test 
machines 23, the dispatcher 17 sends a command to installer 
20 35 which tells installer 35 the name and IP address of the test 
machine which is to have an operating system installed on 
it as well as the type or version of operating system to be 
installed. As stated above, the installer 35 is connected to the 
RS232 ports of each of the test machines 23 via multiplexer/ 
25 demultiplexer 108. Multiplexer/demultiplexer 108 provides 
two-way communications between the test machines 23 and 
the installer 35. 

Each installer 35 contains a list of the dispatchers 17 with 
which it can communicate. Each installer 35 will also 
30 contain a list of the test machines 23 which it can configure 
and/or install. The installer 35 sends this list to the dispatch- 
ers 17 at a periodic rate so that the dispatchers 17 will know 
the installers 35 with which it can communicate. This list 
tells the installer 35 which test machines 23 it can interface 
35 with through the RS232 ports of the test machines. Once an 
installer 35 is activated, it will identify itself to all dispatch- 
ers 17 and then wait for requests from the dispatchers 17. 
Request sent from the dispatchers 17 to the installers 35 will 
be sent in packets in accordance with the communications 



the hardware is configured, an operating system is installed 40 protocol of the present invention. When the installer 35 



on the computer by the user inputting appropriate replies in 
response to the prompts output to the display by the boot 
ROM. 

Another known way of configuring the boot ROM of a 



receives a request from a dispatcher 17 to configure or install 
an OS on a test machine 23, the installer 35 will send 
commands over the Internet 109 to the multiplexer/ 
demultiplexer 108, which demultiplexes the commands and 

computer and of installing an operating system on the 45 inputs them into the RS232 port of the correct test machine, 

computer is to use a cold installer such as Hewlett Packard When the command is received by the test machine 23, the 

Ignite UX. Ignite UX is an installer which can be used to boot ROM on the test machine will output requests to its 

remotely configure and install a computer by allowing a RS232 port which will be multiplexed by multiplexer/ 

person to interact with the boot ROM of the computer over demultiplexer 108 and sent to the installer 35. The installer 

the LAN ethernet port of the computer when the computer 50 35 and the test machine 23 will communicate back and forth 

is connected to a LAN. In accordance with one embodiment in a manner analogous to the manner in which a user 

of the present invention, the automated testing system of the inputting data into the console of a PC replies to the boot 

present invention uses Ignite UX to automatically configure ROM prompts sent to the display to configure the boot ROM 

and install test machines be calling a script which causes settings and install an operating system. In accordance with 

Ignite UX to automatically configure and/or install the test 55 the preferred embodiment of the present invention, when the 

machine. It will be understood to those skilled in the art, in installer 35 is instructed by the dispatcher 17 to install an OS 

view of the discussion of the automated testing system of the on a test machine 23, the installer 35 calls a script which is 

present invention provided herein, the manner in which this well defined to the system. This script is provided with the 

can be accomplished. name and IP address of the test machine to be installed, the 

In accordance with a preferred embodiment of the present 60 OS version and cycle to be installed, and a dispatcher IP 

invention, a novel method and apparatus has been provided address. The script then causes the OS to be installed on the 

which allows a test machine 23 to be automatically config- selected test machine. 

ured and/or have an operating system installed on the test Once the boot ROM settings have been configured and the 

machine 23 by connecting the RS232 ports of the test OS has been installed on the test machine, the launcher is 

machines to the network and configuring and installing the 65 installed on the test machine. The reason that the script is 

test machines 23 via the RS232 ports (not shown) of the test provided with the dispatcher IP address is to allow the 

machines 23. In accordance with this embodiment, the launcher to connect with the dispatcher that initiated the 
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installment. Although the launchers are capable of commu- desired criterion and performed in virtually any desired 

mealing with more than one dispatcher, the launchers only manner. Those skilled in the art will understand how such 

communicate with the dispatchers one at a time and in a tasks may be performed in view of the description provided 

round robin sequence. Therefore, the first dispatcher which herein. Scheduling may also take into account the scarcity of 

is contacted by the launcher to request work is the dispatcher 5 the resources required for performing the job. For example, 

which initiated the installment. If that dispatcher does not if only one test machine exists which is capable of perform- 

have a job for the test machine to execute, the launcher ing certain types of tests, the dispatcher may take into 

moves to the next dispatcher in the sequence to request account the length of time required to perform each of the 

work. jobs in determining the order of jobs. Also, certain users or 

As stated above, users of the automated testing system of 10 groups of users may have higher priorities than other users 

the present invention can add their own computers to the or groups of users and these priorities may be taken into 

pool of computers being used as test machines. Looking account. 

again at FIG. 1, a user accomplishes this by sending a It will be apparent to those skilled in the art that the 

request from his computer 5 via the requester 13 to the automated testing system of the present invention is not 

dispatcher 17 notifying the dispatcher that the user's com- 15 limited with respect to the embodiments discussed above 

puter 5 is to be placed in the test machine pool to be used as and that other features may be added to or removed from the 

a test machine. At this point, the user's computer 5 is already system without deviating from the spirit and scope of the 

configured and has an operating system installed thereon. present invention. It will be apparent to those skilled in the 

Therefore, in most cases, the user's computer 5 will only art that certain features of the automated testing system of 

need to have a launcher 35 installed on it. In some cases, 20 me present invention are not necessary but are merely 

however, the user's computer 5 may need to be configured desirable. It will also be apparent to those skilled in the art 

and/or have an operating system installed on it. When it is that tasks which are performed by certain machines of the 

determined that the user's computer 5 needs to have an automated testing system may be off-loaded onto other 

operating system installed on it, the dispatcher 17 sends a machines of the automated testing system if deemed desir- 

command to the installer 35 which tells the installer 35 to 25 a bl e in order to improve performance and/or maximize the 

send a command to the RS232 port or to the LAN port of the efficiency of the system, 

user's computer 5 which tells the computer 5 to dump its It will also ^ apparent to those skilled in the art that the 

operating system and go into its boot ROM. The user's packets of information transmitted between the users and the 

computer 5 then dumps its operating system and goes into its components of the automated testing system and among the 

boot ROM. The installation script discussed above is then 30 com ponents of the automated testing system can contain any 

called and told which computer to configure and/or install. desired m f ormationi It wiU a is 0 be apparent to those skilled 

The configuration and/or installation process is then carried in the art thatj although and ASCII-based communications 

out in the manner discussed above. Once the computer has protocol is ^ with the automat ed testing system of the 

been configured and installed, the script causes a launcher 18 present inV ention in accordance with the preferred 

to be installed on the computer 5. The launcher then con- 35 embodiment> other communications protocols can also be 

nects to the dispatcher that initiated the installation. used with the autom ated testing system of the present 

It should be noted that if the user's computer 5 does not invention, 

need to be reconfigured or have an operating system ^ a|so be m to those fa the M m 

installed on U, the dispatcher will only cause a launcher to aItb h co erits of the aulomated testi 

be installed oc i the computer 5. Also the dispatchers mam- « tem of tbe , mvemion have ^ discussed ^ 

tain a Ust of the test machines which may be reconfigured implemented in lbese exponents may also be 

and/or have operating systems installed on them Therefore, implemented in hardware and/or in a combination of hard- 

once a test machine ls added to the test machine pool, a wafe and jj M desired Likewjse ^ nems 

dispatcher will only cause the machine to be reconfigured of , be , mvention wbich bave ^ disc)Jssed £ w 

and/or installed if the test machine is listed as one which can « . lememed m hardware ^ „ e . lemented £ 

be reconfigured and/or installed. This information is pro- software and/or fa , combiDation of hardware and soflware . 

vided to the dispatcher by the launcher upon installation of j, ^ 5e , to , bose skiUed m , he m tha , omer 

the launcher. The launchers provide the dispatchers with modiflcations may be made t0 me automated testing systen 

information regarding whether a new OS can be installed on of , he t mvell|ion discussed above which ,£ witbiD 

the test machine, whether software other than the launcher 50 ^ ^ rf ^ ( ^ whic 

can be installed on the test machine, whether files contained therefore, are intended t0 te covered 5y lhe claims M forth 

in the test machine can be written or modified, and which hereinafter 

users or groups of users will have access to the test machine. W hat is claimed is* 

Therefore, when a user decides to place his or her computer t ^ automated testi tem comprising: 

in the test machine pool, the user can place limitations on the " , c „ . 

use of the test machine. a P luralltv of ^ r interfaces, each user interface display- 
ing test parameter choices to a user from which the user 

VII. Conclusion m ay select test parameters relating to a test to be 

It will be apparent to those skilled in the art that the performed, each user interface generating request data 

operations described above with respect to FIGS. 1-8 are 60 packets in response to selections from the user and 

merely intended to demonstrate the preferred embodiment of outputting the request data packets onto the network, 

the automated testing system of the present invention and &e request data packets comprising information relat- 

the preferred functionalities of the different components of ing to test parameters selected by the user and com- 

the automated testing system of the present invention. It will mands indicating that performance of a test is being 

be apparent to those skilled in the art that other features can 65 requested; 

be added if so desired. For example, prioritizing and sched- at least one dispatcher machine in communication with 

uling the tests can be accomplished based on virtually any the user interfaces via the network, the dispatcher 
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machine receiving the request data packets over the 
network and maintaining a list of tests to be performed; 

a plurality of test machines in communication with the 
dispatcher machine via the network, one or more of the 
test machines generating availability data packets, each 
availability data packet indicating that the test machine 
which generated the availability data packet is available 
to perform a test, the availability data packets being 
sent over the network to the dispatcher machine, 
wherein the dispatcher machine selects a test machine 
for performing a test in response to receiving an 
availability data packet; 

a library in communication with the user interfaces and 
with the test machines via the network, wherein a user 
can query the library via the user interface for infor- 
mation relating to tests and select test parameters 
relating to a test to be performed based on the infor- 
mation provided by the library to the user interface, 
wherein when a test machine is instructed by the 
dispatcher machine to perform one of the tests deter- 
mined to be capable of being performed by the tests 
machine, the test machine instructed to perform the test 
obtains the test to be performed from the library and 
executes the test; and 

a plurality of launcher programs, at least one launcher 
program installed on each one of the test machines 
which provides an interface between the test machines 
and the dispatcher machine and between the library and 
the test machines, wherein the launcher program pre- 
pares the test machine on which the launcher program 
is installed to execute a test, obtains tbe test and any 
archives associated with the test from the library, 
installs the test and any associated archives on the test 
machine, prepares the test and archives to be executed, 
causes tbe test machine to execute the test, and provides 
the results of the executed test to the dispatcher 
machine, 

wherein the dispatcher machine determines whether any 
of the tests on the list are capable of being performed 
by the test machine which generated the availability 
data packet, wherein if the dispatcher machine deter- 
mines that one or more of the tests on the list are 
capable of being performed by the test machine which 
generated that availability data packet, the dispatcher 
machine instructs the test machine which generated the 
availability data packet to perform one of the tests 
determined to be capable of being performed on the test 
machine. 

2. The testing system of claim 1, wherein the network 
comprises the Internet and wherein the user interfaces, the 
dispatcher machine and the test machines are each identified 
by an Internet Protocol address, each of the packets gener- 
ated by the user interfaces, the dispatcher machine and the 
test machines including the Internet Protocol address of a 
location to which the packet is being sent. 

3. The testing system of claim 1 further comprising: 

a multiplexer/demultiplexer having an interface con- 
nected to a port of each of the test machines and having 
an interface connected to the network; 

an installer machine in communication with the dis- 
patcher machine via the network, wherein the installer 
machine receives an installation data packet from the 
dispatcher indicating that a particular test machine is to 
have an operating system installed on it and, in 
response to receiving the installation data packet, 
causes an operating system to be installed on the 



particular test machine via the port of the particular test 
machine prior to instructing the test machine to perform 
the test. 

4. The testing system of claim 1, wherein each launcher 
5 program sends information to the dispatcher machine indi- 
cating to the dispatcher machine which users can use the test 
machine hosting the launcher and wherein the dispatcher 
machine maintains a permissions file which the dispatcher 
machine uses to determine whether a particular user can use 

10 a particular test machine. 

5. The testing system of claim 3, wherein the installer 
machine is capable of configuring boot ROM settings on the 
test machines, wherein the installer machine receives a 
configuration data packet from the dispatcher machine indi- 

15 eating that the boot ROM settings on a particular test 
machine are to be configured and, in response to receiving 
the configuration data packet from the dispatcher machine, 
causes the boot ROM settings on the particular test machine 
to be configured via the port connected to the multiplexer/ 

2 q demultiplexer. 

6. The testing system of claim 3, wherein the installer 
machine is capable of causing operating systems to be 
simultaneously installed on a plurality of the test machines. 

7. The testing system of claim 4, wherein each launcher 
program includes a list of the dispatcher machines with 
which it is allowed to communicate, and wherein the 
launcher program only sends availability data packets to 
dispatcher machines which are on the list of dispatcher 
machines with which it is allowed to communicate. 

8. Tbe testing system of claim 5, wherein the installer 
machine is capable of causing boot ROM settings to be 
simultaneously configured on a plurality of the test 
machines. 

9. An automated testing system distributed over the 
Internet, the automated testing system comprising: 

a plurality of user interfaces, each user interface having an 
Internet Protocol address, each user interface display- 
ing test parameter choices to a user from which the user 
may select test parameters relating to a test to be 
performed, each user interface generating request data 
packets in response to selections from the user and 
outputting the request data packets onto the Internet, 
the request data packets comprising information relat- 
ing to test parameters selected by the user, an Internet 
Protocol address of a location to which the request data 
packet is being sent, and command information indi- 
cating that performance of a test is being requested; 
a plurality of dispatcher machines, each dispatcher 
machine being located at an Internet Protocol address 
contained in one of the request data packets, each 
dispatcher machine receiving the request data packets 
containing the Internet Protocol address designating the 
dispatcher machine and maintaining a list of tests to be 
performed; 

a plurality of test machines, each test machine having its 
own Internet Protocol address, one or more of the test 
machines generating availability data packets and out- 
putting them onto the Internet, each availability data 
packet containing the Internet Protocol address of the 
test machine which generated the availability data 
packet and indicating that the test machine identified by 
the Internet Protocol address contained in the availabil- 
ity data packet is available to perform a test, each 
availability data packet containing the Internet Protocol 
address of one of the dispatcher machines, wherein the 
dispatcher machine identified by the Internet Protocol 
address contained in an availability data packet 



25 



30 



35 



40 



45 



50 



65 



03/31/2004, EAST Version: 1.4.1 



6,163,805 

21 22 

receives the availability data packet and selects a test 12. The testing system of claim 11, wherein each test 

machine for performing a requested test in response to machine has a launcher program installed thereon which 

receiving an availability data packet; provides an application program interface between the test 

a multiplexer/demultiplexer having an interface con- machines and the dispatcher machine and between the 

nected to a port of each of the test machines and having 5 library and the test machines, wherein the launcher program 

an interface connected to the network; and prepares the test machine on which the launcher program is 

an installer machine in communication with the dis- installed to execute a test, obtains the test and any archives 

patcher machine via the network, wherein the installer associated with the test from the library, installs the test and 

machine receives an installation data packet from the any associated archives on the test machine, prepares the test 

dispatcher indicating that a particular test machine is to 10 and archives to be executed, causes the test machine to 

have an operating system installed on it and, in execute the test, and provides the results of the executed test 

response to receiving the installation data packet, to the dispatcher machine. 

causes an operating system to be installed on the 13, The testing system of claim 9, wherein the installer 

particular test machine via the port of the particular test machine is capable of configuring boot ROM settings on the 

machine prior to instructing the test machine to perform test machines, wherein the installer machine receives a 

the test, wherein when a particular dispatcher machine configuration data packet from the dispatcher machine indi- 

receives an availability data packet generated by a eating that the boot ROM settings on a particular test 

particular test machine, the particular dispatcher machine are to be configured and, in response to receiving 

machine determines whether any of the tests on the list the configuration data packet from the dispatcher machine, 

maintained by the particular dispatcher machine are causes the boot ROM settings on the particular test machine 

capable of being performed by the particular test to be configured via the port connected to the multiplexer/ 

machine, wherein if the particular dispatcher machine demultiplexer. 

determines that one or more of the tests are capable of 14. The testing system of claim 9, wherein the installer 

being performed by the particular test machine, the machine is capable of configuring boot ROM settings on the 

dispatcher machine instructs the particular test machine test machines, wherein the installer machine receives a 

to perform one of the tests determined to be capable of configuration data packet from the dispatcher machine indi- 

being performed on the particular test machine. catm g that the boot ROM settings on a particular test 

10. The testing system of claim 9, wherein the network machine are to be configured and, in response to receiving 
comprises the Internet and wherein the user interfaces, the ^ the configuration data packet from the dispatcher machine, 
dispatcher machine and the test machines are each identified causes the boot ROM settings on the particular test machine 
by an Internet Protocol address, each of the packets gener- to be configured via the ethernet local area network port of 
ated by the user interfaces, the dispatcher machine and the the particular test machine. 

test machines including an indication of the Internet Proto- 15. The testing system of claim 14, wherein each launcher 

col address of a location to which the packet is being sent. ^ program sends information to the dispatcher machine indi- 

11 . The testing system of claim 9 further comprising: eating to the dispatcher machine which users can use the test 
a library in communication with the user interfaces and machine hosting the launcher and wherein the dispatcher 

with the test machines via the network, wherein a user machine maintains a permissions file which the dispatcher 

can query the library via the user interface for infor- machine uses to determine whether a particular user can use 

mation relating to tests and select test parameters 4Q a particular test machine. 

relating to a test to be performed based on the infor- 16. The testing system of claim 15, wherein each launcher 

mation provided by the library to the user interface, program includes a list of the dispatcher machines with 

wherein when a test machine is instructed by the which it is allowed to communicate, and wherein the 

dispatcher machine to perform one of the tests deter- launcher program only sends availability data packets to 

mined to be capable of being performed by the tests 45 dispatcher machines which are on the list of dispatcher 

machine, the test machine instructed to perform the test machines with which it is allowed to communicate, 
obtains the test to be performed from the library and 

executes the test. ***** 



03/31/2004, EAST Version: 1.4.1 



UNITED STATES PATENT AND TRADEMARK OFFICE 

CERTIFICATE OF CORRECTION 



PATENT NO. : 6,163,805 Page 1 of 1 

DATED : December 19, 2000 

INVENTOR(S) : Stephen Silva et al. 



It is certified that error appears in the above-identified patent and that said Letters Patent is 
hereby corrected as shown below: 



Title page. 

Item [57], ABSTRACT, 

Line 30, after "data packet." insert - If one or more of the tests listed are capable of 
being performed by the available test machine, the dispatcher machine instructs the test 
machine to perform one of the tests, preferably the test having the highest priority. -- 

Column 12, 

Line 7, "IV. User/Reguester/Library Interactions" should read -- IV. User/Requester/ 
Library Interactions -- 

Column 17, 

Line 58, "VII." should read -- VII -- 
Column 2L 

Line 44, delete "tests" and insert therefor - test -- 



Signed and Sealed this 
Tenth Day of December, 2002 




JAMES E. ROGAN 
Director of the United States Patent and Trademark Office 



03/31/2004, EAST Version: 1.4.1 



